Current computing and software paradigms have so far prevented truly scalable neural models that may simulate biology in reasonable amounts of time.
There are two major steps in simulating a neural network such as a nervous system: incrementally solving governing equations in various part of the network or system; and communicating the results to other parts of the network or system.
Exemplary systems may include one or more of the following: M. Mahowald, “VLSI analogs of neuronal visual processing: A synthesis of form and function,” Ph.D. dissertation, California Inst. Technol., Pasadena, Calif., 1992; K. Boahen, “A burst-mode word-serial address-event link-I: transmitter design,” IEEE Transactions on Circuits and Systems I, vol. 51, no. 7, pp. 1269-80, 2004; K. Boahen, “A burst-mode word-serial address-event link-II: receiver design,” IEEE Transactions on Circuits and Systems I, vol. 51, no. 7, pp. 1281-91, 2004; K. Boahen, “Point-to-point connectivity between neuromorphic chips using address events,” IEEE Transactions on Circuits and Systems II, vol. 47, no. 5, pp. 416-34, 2000; Javier Navaridas, Mikel Luján, Jose Miguel-Alonso, Luis A. Plana, and Steve Furber. 2009. Understanding the interconnection network of SpiNNaker. In Proceedings of the 23rd international conference on Supercomputing (ICS '09). ACM, New York, N.Y., USA, 286-295; M. D. Humphries, R. D. Stewart, and K. N. Gurney, “A physiologically plausible model of action selection and oscillatory activity in the basal ganglia,” The Journal of Neuroscience, vol. 26, no. 50, pp. 12921-12942, 2006; C. M. Thibeault, R. Hoang, and F. C. Harris Jr., “A novel multi-gpu neural simulator,” in ISCA's 3rd International Conference on Bioinformatics and Computational Biology (BICoB '11), New Orleans, La., March 2011.
It is known to distribute the simulation of a large neural network. Different portions of the model are then simulated by separate computers or nodes in parallel. The neural model is integrated at each iteration, and the spiking information is sent to all of the neurons connected to those that fired.
Ideally, when parallelizing the simulation of spiking neural networks the computational cost of the mathematical integration and synaptic computations is balanced with cost of communicating information between nodes (single computers within a cluster). Historically, the communication time was significantly lower than the compute time. With the introduction of higher-performance architectures such as General Purpose Graphical Processing Units (GPGPU) and specialized neural hardware systems, this is no longer the case. However, the way spiking information is sent has not changed.
Known hardware and software simulation environments use a variant of address event representation (see for example the Boahen, 2000 reference above). The simplest form of such an address event representation is that when a neuron fires an action potential, an ID number unique to the neuron is sent to all of the nodes that contain post-synaptic neurons connected to the one that fired. The addresses of all the neurons that fire during the current iteration can be collected and sent as a single packet to all of the connected nodes.
As the number of neurons that fired increases, however, the size of the address packets correspondingly increases. In this case, the time spent in communication is a direct correlation to the number of neurons that fired. Similarly, as the number of compute nodes increases so does the number of packets that need to be sent. In some cases, for both software and hardware based systems, this can prevent scaling up to desirable model sizes.
The present disclosure relates to systems and methods for preventing the size of the address packets to increase as in the known schemes described above.